Title of dissertation : STATISTICAL ANALYSIS OF AUTOMATIC SUMMARIZATION EVALUATION
نویسندگان
چکیده
Title of dissertation: STATISTICAL ANALYSIS OF AUTOMATIC SUMMARIZATION EVALUATION Peter A. Rankel, Doctor of Philosophy, 2016 Dissertation directed by: Professor Eric V. Slud Department of Mathematics Dr. John M. Conroy IDA/Center for Computing Sciences This dissertation applies statistical methods to the evaluation of automatic summarization using data from the Text Analysis Conferences in 2008-2011. Several aspects of the evaluation framework itself are studied, including the statistical testing used to determine signi cant di erences, the assessors, and the design of the experiment. In addition, a family of evaluation metrics is developed to predict the score an automatically generated summary would receive from a human judge and its results are demonstrated at the Text Analysis Conference. Finally, variations on the evaluation framework are studied and their relative merits considered. An over-arching theme of this dissertation is the application of standard statistical methods to data that does not conform to the usual testing assumptions. STATISTICAL ANALYSIS OF AUTOMATIC SUMMARIZATION EVALUATION
منابع مشابه
A survey on Automatic Text Summarization
Text summarization endeavors to produce a summary version of a text, while maintaining the original ideas. The textual content on the web, in particular, is growing at an exponential rate. The ability to decipher through such massive amount of data, in order to extract the useful information, is a major undertaking and requires an automatic mechanism to aid with the extant repository of informa...
متن کاملSummarization Evaluation : Correlating Human Performance on an Extrinsic Task with Automatic Intrinsic Metrics
Title of dissertation: Text Summarization Evaluation: Correlating Human Performance on an Extrinsic Task with Automatic Intrinsic Metrics Stacy F. Hobson Doctor of Philosophy, 2007 Dissertation directed by: Professor Bonnie J. Dorr Department of Computer Science Text summarization evaluation is the process of assessing the quality of an individual summary produced by human or automatic methods....
متن کاملAnalysis of Summarization Evaluation Experiments
The goals of my dissertation are: 1) to propose a French terminology for the presentation of evaluation results of automatic summaries, 2) to identify and describe experimental variables in evaluations of automatic summaries, 3) to highlight the most common tendencies, inconsistencies and methodological problems in summarization evaluation experiments, and 4) to make recommendations for the pre...
متن کاملA Multi-document Summarization System for Sociology Dissertation Abstracts: Design, Implementation and Evaluation
The design, implementation and evaluation of a multi-document summarization system for sociology dissertation abstracts are described. The system focuses on extracting variables and their relationships from different documents, integrating the extracted information, and presenting the integrated information using a variable-based framework. Two important summarization steps – information extrac...
متن کاملLetSum, an Automatic Text Summarization system in Law field
This paper presents our work on the development of a new methodology for automatic summarization of justice decision. We describe LetSum (Legal text Summarizer), a prototype system, which determines the thematic structure of a judgment in four themes Introduction, Context, Juridical Analysis and Conclusion. Then it identifies the relevant sentences for each theme. We have carried our an evaluat...
متن کامل